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ABSTRACT 


The Barriers to Seeking Psychological Help Scale (BSPHS) was originally developed to measure factors that kept 
college students from seeking mental health assistance. However, the underutilization of mental health services 
among adults is also common, and understanding the factors that inhibit adults from seeking help for mental health 
issues is necessary. One important first step to increasing the use of mental health services among this population 
is to identify the barriers preventing them from using such services through valid and reliable scales. However, no 
validated and reliable scale currently exists in the Turkish language to measure the barriers to seeking 
psychological help among adults. Thus, the purpose of this cross-sectional study was to examine the underlying 
factor structure and reliability of the BSPHS among Turkish adults. A convenience sample of 208 (134 female, 74 
male, M=35.58) Turkish adults completed the BSPHS and a demographic information form. Confirmatory factor 
analyses were performed to test the underlying factor structure of the BSPHS using three competing models. 
Reliability analysis was also used to examine mean inter-item correlation and item-total correlations, as well as to 
estimate Cronbach alpha reliability. The results of this study suggest that the BSPHS has a similar five-factor 
structure as suggested by Topkaya, Sahin, and Meydan (2017). Reliability analyses also suggested that all 
subscales of the BSPHS had adequate mean inter-item correlation, item-total correlations, as well as Cronbach 
alpha reliability. Therefore, the BSPHS can be used to measure the factors that inhibit Turkish adults from seeking 
psychological help. Future studies can examine convergent, divergent, and predictive validity and test-retest 
reliability of the BSPHS among Turkish adults. 

Keywords: Barriers to seeking psychological help scale, confirmatory factor analysis, reliability, Turkish adults. 


INTRODUCTION 


Help-seeking is a term generally used to refer to the “behavior of actively seeking help from other people” 
(Rickwood, Deane, Wilson, & Ciarrochi, 2005). Help-seeking can be informal (e.g., trying to solve the problem 
on their own or seeking help from friends and family) and/or formal (e.g., seeking help from mental health 
professionals, college counseling centers, teachers) (Rickwood et al., 2005). Meta-analytic evidence suggests the 
efficacy of psychological therapies and counseling for treating mental disorders (Lipsey & Wilson, 1993; Shadish, 
Navarro, Matt, & Phillips, 2000). On the other hand, not seeking help or delaying help-seeking can result in 
negative health indicators, such as lower quality of life, substance abuse, engaging in risky sexual behavior, and 
premature death (Anderson & Lowen, 2010; Brindis et al., 2007; Brindis, Park, Ozer, & Irwin, 2002; Laski, 2015). 
Empirical evidence also suggests that distressed people receiving empirically supported therapies commonly 
experience increases in well-being, happiness, and quality of life and decreases in perceived life stress and negative 
mental health outcomes (Spring, 2007; Wampold & Imel, 2015). Although the benefits of help-seeking have been 
well documented in the literature, as well as the high prevalence of mental health problems reported among the 
general population in Turkey (Erol, Kilig, Ulusoy, Kegeci, & Simsek, 1998), the rates of formal help-seeking for 
psychological problems are rather low among Turkish adults (Topkaya, 2015a, 2015b). Therefore, understanding 
the factors that inhibit Turkish adults from seek mental health help is necessary. 


A possible initial step to increase the use of mental health services among this population is to identify barriers 
inhibiting them from using such services through validated and reliable scales. However, there is currently no 
validated and reliable measure in the Turkish language that can be used to measure the barriers to seeking 
psychological help among adults. The Barriers to Seeking Psychological Help Scale (BSPHS) was originally 
developed to measure the factors that keep college students from seeking mental health assistance and may also 
be used to screen barriers to seeking psychological help among adults. However, since its development, its factor 
structure has rarely been examined in different populations, such as Turkish adults. This is concerning because the 
barriers to seeking psychological help may be different and the proposed factor structure may not be the same for 
Turkish adults. Consequently, using the BSPHS subscale scores for Turkish adults may lead to erroneous 
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conclusions without having observed consistent relationships between the BSPHS subscale items and related latent 
variables (Horn & McArdle, 1992). 


The BSPHS consists of 17 items and five subscales, namely fear of being stigmatized by society, trust in the mental 
health professional, difficulties in self-disclosure, perceived devaluation, and lack of knowledge. The fear of being 
stigmatized by society subscale measures irrational beliefs clients hold related to seeking psychological help in 
society. Trust in the mental health professional measures clients’ concerns related to the mental health professional. 
Difficulties in self-disclosure measures clients’ perceived difficulties disclosing their problems to a mental health 
professional. The perceived devaluation subscale measures self-derogatory beliefs related to seeking psychological 
help. Lastly, the lack of knowledge subscale measures lack of information about seeking psychological help. In 
the initial development of the scale across five studies, researchers examined the validity and reliability of the 
BSHPS and reported evidence for construct, convergent, and discriminant validity, as well as test-retest reliability 
and internal consistency. 


Specifically, the BSPHS demonstrated a five correlated factor structure using exploratory factor analysis (EFA) in 
a college student sample. The fear of being stigmatized subscale was composed of four items and explained 34.24% 
of the total variance. This subscale’s factor loadings, based on EFA, ranged from .74 to .94. The trust in the mental 
health professional subscale also consisted of four items and explained 10.30% of the total variance. Its factor 
loadings ranged from .31 to .81. Difficulties in self-disclosure was composed of three items and explained 8.33% 
of the total variance. The factor loadings ranged from .64 to .87. The perceived devaluation subscale also consisted 
of three items and explained 7.08% of the total variance, with factor loadings ranging from .40 to .86. Lastly, the 
lack of knowledge subscale consisted of three items and explained 6.71% of the total variance. Its factor loadings 
ranged from .33 to .95. Confirmatory factor analysis (CFA) using the robust maximum likelihood estimate also 
supported a five correlated factor structure in a cross-validation sample. Cronbach alpha internal consistency 
reliability estimates for the BSPHS subscales across two studies were also good, ranging from .58 (lack of 
knowledge) to .91 (fear of being stigmatized by society). The three-week stability coefficient or test-retest 
reliability was also good, ranging from .56 (perceived devaluation) to .75 (for difficulties in self-disclosure) among 
the subscales. Although researchers found evidence for validity and reliability of the BSPHS, evidence regarding 
the factor structure and reliability among Turkish adults is scarce. Thus, the purpose of this cross-sectional study 
was to examine the factor structure and reliability of the BSPHS among Turkish adults. 


METHOD 


Participants 

A cross-sectional research design was used in this study. Participants were 208 Turkish adults living in the Central 
Black Sea Region of Turkey. They were selected using convenience sampling (Cohen, Manion, & Morrison, 
2018). Specifically, the researchers included their relatives, acquaintances, and friends in the study sample. Then, 
they asked these people to help them find adults from Samsun city who were able to complete the survey. There 
were 134 women (64.4%) and 74 men (35.6%) who participated in the study. The age of the participants ranged 
from 18 to 80 years old, and the mean age was 35.58 years (SD: 13.15). Five participants did not report their age. 


Measures 
Demographics. A personal information form was used to collect information about the participants’ sex and age. 


BSPHS: The barriers to seeking psychological help among Turkish adults was measured by the BSPHS (Topkaya 
et al., 2017). More detailed information about the psychometric characteristics of the BSPHS were given in the 
introduction. Participants indicate their degree of agreement or disagreement with each item on a five-point Likert- 
type scale ranging from strongly disagree (1) to strongly agree (5). Scores can range from 4 to 20 for the fear of 
being stigmatized by society and trust in mental health professional subscales and from 3 to 15 for difficulties in 
self-disclosure, perceived devaluation, and lack of knowledge subscales. Higher scores reflect higher levels of 
perceived barriers in each dimension. 


Procedure 

Data was collected between January and March 2019. Participants were informed about the study’s purpose and 
ethical considerations. Specifically, the researchers told them that participation in the study was voluntary, the 
answers would remain anonymous, and the participants could withdraw from the study without any consequences. 
Written informed consent was also obtained from all participants prior to completing the questionnaires. All adults 
voluntarily participated in the study and completed the questionnaires in approximately 10 minutes. 


Statistical Analysis 
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All statistical analyses were performed using SPSS 23 and Mplus 7.0 (Muthén & Muthén, 1998-2015). All CFA 
models were estimated using the mean and variance adjusted maximum likelihood estimator (MLMV) taking into 
account of the use of ordered-categorical variables with five response categories (e.g., Likert-type scales) in this 
study (Finney & DiStefano, 2013; Rhemtulla, Brosseau-Liard, & Savalei, 2012). The MLMV produces parameter 
estimates and standard errors, as well as mean and variance adjusted chi-square statistics that are robust to non- 
normality (Muthén & Muthén, 1998-2015). Ina recent extensive simulation study, Maydeu-Olivares (2017) found 
that the MLMV estimator is the optimal choice among different ML estimators across different normal and non- 
normal distributions with accurate Type I and standard errors. Three competing models were tested in this study: 
a single-factor model (Model 1), a correlated five-factor model (Model 2), and a correlated five-factor model that 
permits residual variances to be correlated based on substantive values or previous studies (Model 3). A schematic 
presentation of all competing models tested in the present study can be found in Figure 1. Assessment of model fit 
was based on multiple goodness of fit statistics: the adjusted Chi-Square (y7/df), Root Mean Square Error of 
Approximation (RMSEA) with its 90% confidence interval and significance level, Comparative Fit Index (CFD, 
Tucker-Lewis Index (TLI), and standardized root mean square residual (SRMR). Although no absolute standards 
exist for cut-off values for goodness of fit indices in CFA, values lower than 5 but higher than 3 typically indicate 
adequate model fit for y° /df, and values smaller than 3 indicate excellent fit. Values smaller than .08 or .06 for the 
RMSEA indicate adequate and excellent model fit, respectively. Values greater than .90 and .95 for the CFI and 
TLI indicate adequate and excellent model fit, respectively. Lastly, a SRMR value close to or less than .08 indicates 
a good fit to the data (Byrne, 2012; Gana & Broc, 2019; Keith, 2019; Wang & Wang, 2013). Reliability analyses 
were performed using mean inter-item correlation, item-total correlation, and Cronbach alpha internal consistency 
coefficient. The data that support the findings of this study are available in Open Science Framework 
(osf.10/hnfv4). 
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Model 3 


Note: v=item, F1=Fear of being stigmatized by society, F2=Trust in the mental health professional, F3=Difficulties 
in self-disclosure, F4=Perceived devaluation, F5=Lack of knowledge in Model 2 and Model 3. 


RESULTS 

A series of CFAs were conducted to examine competing models among Turkish adults. Goodness of fit indices 
for the competing models are shown in Table 1. As seen in Table 1, the goodness of fit values for the single-factor 
model were poor (Model 1). The five correlated factor model, as suggested by Topkaya et al. (2017) for 
undergraduate students, had an acceptable fit to data (Model 2). However, inspection of local fit via normalized 
residual covariances and modification index for Model 2 suggested releasing error covariances between item 4 
and item 12. Both items are related to worries about the mental health professional and from the trust of the mental 
health professional subscale. 


Table 1: Goodness of fit indices for competing models 


7 df ldf CFI TLI RMSEA Dp RMSEA 90% SRMR 
Confidence Interval 
Model 1 287.425 119 2.415 .765 .731 .082 .001* .070-.095 .078 
Model 2 164.081 109 1.505 .923 .904 .049 514 .033-.064 .062 
Model 3 149.281 108 1.382 .942 .927 .043 .755 .024-.059 .056 


Note: p<.001. 


Thus, Model 3 estimated adding item 4 and item 12 to correlated errors. As seen in Table 1, Model 3 was the best 
fitting model to the data. Most goodness of fit indices were excellent. Table 2 shows standardized item factor 
loadings, item standard errors, z-values, latent factor correlations, and squared multiple correlation of items (R’). 
The estimated R? values give information about how much variance of each observed BSPHS item was explained 
by the factor that item is loaded on and also equivalent to the squared standardized item factor loading for each 
item. All factor loadings and latent factor correlations were large and statistically significant at least at p<.001. 
The estimated R? values also ranged from medium to large effect sizes. 
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Table 2: Results of confirmatory factor analysis 


Item No/Item nN S.E Zz R? 


Fear of being stigmatized by society (F1) 


2. I worry about being stigmatized as “problematic” and/or “crazy” if I seek .630 .060 10.501 .397 
psychological help. 

6. I worry about what other people would think about me ifI seek psychological .830 .033 25.120 .688 
help. 

9. I worry about whether my friends would mock me if I seek psychological 796 .036 «21.932 633 
help. 


14. I don’t want to seek psychological help as it’s not accepted as .740 .038 19.409 .548 
“normal/natural” in the culture in which I grew up. 
Trust in the mental health professional (F2) 


4. I don’t trust professionals to keep my issues confidential. 647 .056 11.454 418 
12. I worry that the professional wouldn’t understand me. 767 = =.041) 18.596 589 
16. I worry about whether the professional would listen to me adequately. 689 .039 17.646 .474 


17. I worry that the professional would be insensitive to my problems, as s/he .632 .053 11.910 .399 
constantly meets people with similar problems. 

Difficulties in self-disclosure (F3) 

1. I have difficulty in sharing my problems with a stranger even thoughhe isa .664 .058 11.460 .441 
professional. 

5. I feel ashamed to tell my problems to the professional giving psychological .812 .043 18.989 .659 
help. 

8. I refuse to give information about my private problems (sex, violence, etc.), .738 .045 16.589 .545 
even to a professional. 

Perceived devaluation (F4) 

11. I would feel weak if I told my problems to a professional. .770 = .049.- 15.713 593 
13. My self-confidence might decrease if I seek psychological help. 875.035 25.111 .766 
15. I worry that if I take psychological help once, then I would need it whenever .532  .056 9.497  .283 
I have a problem. 

Lack of knowledge (F5) 

3. I don’t want to spend time seeking psychological help as it would take too 464 076 6.144 .216 
long. 

7. I don’t know how to contact professionals who provide psychological help. 535 .069 7.717 .287 
10. I don’t want to seek psychological help as places that provide such services .660 .067 9.848  .436 
are far away. 


Latent Factor Correlations Fl F2 F3 F4 

Fl 

F2 .610 

F3 636 657 

F4 692.622 577 

F5 653.635 572.606 
Note: }=Item factor loading. All factor loadings and latent factor correlations were statistically significant at least 
at p<0.001. 


The reliability of the BSPHS was analyzed by means of mean inter-item correlation, item-total correlations, and 
Cronbach alpha internal consistency coefficient. The results of the reliability analyses are shown in Table 3. 
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Table 2: Results of BSPHS Reliability Analyses 


r 
Fear of being stigmatized by society 

BSPHS2 562 
BSPHS6 .746 
BSPHS9 .697 
BSPHS14 .641 
Mean inter-item correlation .553 
Cronbach alpha (a) 831 
95% CI Cronbach alpha .790-.866 
Trust in the mental health professional 

BSPHS4 457 
BSPHS12 483 
BSPHS16 .663 
BSPHS17 619 
Mean inter-item correlation 438 
Cronbach alpha (a) .756 
95% CI Cronbach alpha .697-.806 
Difficulties in self-disclosure 

BSPHS1 579 
BSPHSS5 .664 
BSPHS8 .609 
Mean inter-item correlation 543 
Cronbach alpha (a) .778 
95% CI Cronbach alpha .720-.825 
Perceived devaluation 

BSPHS11 591 
BSPHS13 .697 
BSPHS15 453 
Mean inter-item correlation 504 
Cronbach alpha (a) 747 
95% CI Cronbach alpha .681-.801 
Lack of knowledge 

BSPHS3 261 
BSPHS7 299 
BSPHS10 497 
Mean inter-item correlation .283 
Cronbach alpha (a) 532 
95% CI Cronbach alpha .410-.632 


Note: r= corrected item-total correlation, CI: Confidence interval. 


As seen in Table 3, all subscales had an adequate mean inter-item correlation that ranged from .283 (lack of 
knowledge) to .553 (fear of being stigmatized by society). The item-total correlations of the subscales were also 
adequate, with levels ranging from .562 to .746 in fear of being stigmatized by society, .457 to .663 in trust in the 
mental health professional, .579 to .664 in difficulties in self-disclosure, .453 to .697 in perceived devaluation, and 
.261 to .497 in the lack of knowledge subscale. Cronbach alpha reliability estimates also ranged from .532 (lack 
of knowledge) to .831 (fear of being stigmatized by society). 


DISCUSSION 

As underutilization of mental health services among adults is common, it is important to understand the factors 
that inhibit adults from seeking psychological help. However, there is currently no available validated and reliable 
measure in the Turkish language that can be used to measure the barriers to seeking psychological help among 
adults. Thus, this study examined the underlying factor structure of BSPHS items and its reliability among Turkish 
adults. Specifically, this study tested one general factor model, a five correlated factor model, and a five correlated 
factor model with correlated errors in Turkish adults using confirmatory factor analyses. The results of the 
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confirmatory factor analyses found no support for the one-factor model, indicating that a common underlying 
factor was not adequate to account for the pattern of covariance across BSPHS items. There was adequate support 
for the correlated five-factor model and almost excellent support for the correlated five-factor model with 
correlated errors terms. The study findings are in line with Topkaya et al.’s (2017) study among undergraduate 
students showing that the five correlated factor model best fit the BSPHS. These findings suggest that the BSPHS 
may be used for assessing barriers related to seeking help for mental health issues among Turkish adults 


The results of this study also suggest that the subscales of the BSPHS had a mean inter-item correlation ranging 
from .283 (lack of knowledge) to .553 (fear of being stigmatized by society). According to Clark and Watson 
(1995), average inter-item correlation is to be in the range of .15 to .50, but it could even be higher if the construct 
of interest is narrowly defined. All subscales of the BSPHS had a mean inter-item correlation within or above this 
criterion, suggesting that the subscales contain items that are particularly intercorrelated and measure different 
aspects of the construct of interest. The item-total correlation values for each item were also above .20 in each 
subscale, indicating good discrimination, such that higher scores on the item are associated with higher scale scores 
in each subscale and each item is a good indicator of the construct of interest (Meyers, Gamst, & Guarino, 2013). 
All subscales also had a Cronbach alpha coefficient above .70, except for the lack of knowledge subscale. 
However, the internal consistency reliability estimate of the lack of knowledge subscale was similar to that in 
Topkaya et al. (2017). According to DeVellis (2017), measurement tools with a Cronbach alpha reliability 
coefficient of .70 or above can be used for screening and research purposes. Thus, the BSPHS may also be used 
for screening and research purposes, with the notable exception of the lack of knowledge subscale. However, 
considering that the Cronbach alpha reliability coefficient is related to the number of items in a scale and mean 
inter-item correlation (DeVellis, 2017), the relatively low reliability of the lack of knowledge subscale may be 
related to the limited number of items in the subscale as the mean inter-item correlation was adequate (Clark & 
Watson, 1995). Overall, the findings of this study suggest that the BSPHS is a valid and reliable scale that can be 
used to measure factors that inhibit Turkish adults from seeking mental health assistance. 


Finally, this study has some limitations. First, only a limited number of Turkish adults from a specific region of 
Turkey were used to test the underlying factor structure of the BSPHS. Thus, the external validity of this study is 
low. Future studies should examine the factor structure of the BSPHS using a more representative sample of 
Turkish adults. Second, this study only examined the construct validity of the BSPHS. Futures studies should also 
look at the convergent, divergent, and predictive validity of the BSPHS among Turkish adults. Lastly, the 
reliability analyses of the BSPHS were limited to item analyses and Cronbach alpha internal consistency analyses. 
Future studies should also examine short-term and long-term test-retest reliability of the BSPHS. 
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